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Abstract 

The preliminary findings of the evaluation of the training 
program. Creative Problem Solving, are presented. The training 
was provided through the Blumberg Center for Interdisciplinary 
Studies in Special Education at Indiana State University. 
Twenty-one state and county child service coordinators 
participated in the training on a voluntary basis. Participants 
completed a pretest measure, the "Facilitator Self-Evaluation," 
prior to the training and two and four months after the training. 
Statistical significance was obtained on six dependent variables 
on the evaluation instrument. The design of the instrument and 
the threats to the validity of the instrument are discussed. 
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Training State Child Service Coordinators in 
Creative Problem Solving 

Background of the Problem 

Training programs are developed in many disciplines in an 
attempt to provide opportunities for continued learning on the 
job. For example, health care professionals are required to 
participate in continuing education to continue to meet 
certification and licensure requirements (Abrahamson, 1984) . 
Professionals in mental health care and social work participate 
in interdisciplinary training to increase their skills in working 
with particular client and family populations (Sullivan & Clancy, 

1990) . Likewise, teachers and special educators participate in 
professional development programs and in-service training to 
increase their skills in meeting the needs of a diverse group of 
learners (Granlund, Steensson, Sundin, & Olsson, 1992) • 

Evaluation is an important component of such training 
programs; there are several reasons for evaluating the effects of 
training. For example, program developers may seek to determine 
if the program is accomplishing the objectives set out (Phillips, 

1991) . The evaluation may also serve the purpose of identifying 
strengths and weaknesses and improving the content and 
methodology of the training prior to widespread implementation. 

Evaluations of training programs are designed to measure 
different training effects, such as participants' satisfaction 
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with the training, knowledge acquired through the training, or 
changes in behavior due to skill development from the training. 
Researchers have identified several different effects an 
evaluation may be designed to target. Figure 1 is an 
illustration of the possible effects. 




1974, in Patrick, 1992). 



As indicated in Figure 1, an evaluator may measure participants' 
experience of the training (i.e., participants' reactions to the 
training). This may include participants' satisfaction or 
dissatisfaction with the materials, instructors, facility, 
instructional methodology, and/or content of the training. 

While enjoyment of the training is important to the participant, 
this level of measurement does not provide an indicator of 
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learning. Additionally, reactions to a training program are 
often the result of extraneous factors, including the 
participants' own perceived success in the training. 

Beyond measurement of participants' reactions to the 
training, participants' learning can be measured. Participants' 
learning include? their knowledge of the principles, facts, 
techniques, and skills presented in the training. For example, 
it may be that greater awareness of a particular subject is the 
primary goal of the training. Sullivan and Clancy (1990) 
measured participants' ability to identify sexual abuse 
indicators in case studies (the information presented in their 
training) as a measure of program success. In the present study, 
the evaluation measured participants' learning as a result of 
participation in the training. 

The timing of the measurement is important when learning is 
assessed. The program planners should specify specific 
behavioral objectives in advance regarding the length and level 
participants are expected to retain the information presented and 
(Patrick, 1992) . 

The third level of measurement addresses actual behavior 
change that is a result of learning through participation in the 
training. Participants must learn the material presented before 
it is exhibited in their behavior. 

Last, the consequences of the behavior change may also be 
targeted for measurement. In the case of Sullivan and Clancy's 
(1990) research, the learning participants acquired was knowledge 
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Creative Problem Solving 
of sexual abuse indicators. Although not measured in their 
study, measures of participants' behavior subsequent to the 
training might have revealed that participants were more likely 
to identify indicators of abuse in children they served or came 
into contact with. The consequence of this behavior change 
(i.e., the ultimate value) would be the protection of children 
who experience abuse. This level of evaluation has also been 
termed a measure of organizational change and/or organizational 
effectiveness (Phillips, 1991) . 

The instruments commonly used to measure the different 
effects of training include record-keeping systems, 
questionnaires, examinations, attitude surveys, interviews, focus 
groups, and observations (Phillips, 1991). Depending upon the 
design of the evaluation, multiple indices may be used to assess 
the effects of the training. 

The present study is an evaluation of the training project. 
Training Facilitators in Problem Solving in Community-Based Care 
Coordination. The Blumberg Center for Interdisciplinary Studies 
in Special Education at Indiana State University (ISU) developed 
the training project and contracted with the Center for Studies 
in Creativity at Buffalo State College to present the training to 
Indiana state agency employees. 

The impetus and source of funding for the training project 
is the Indiana Collaboration Project, which is a part of the 
Indiana Consolidated Plan for Service to Children and Families. 
One of the goals of the state mandated Indiana Collaboration 
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Project is to "collaboratively manage the delivery of services" 
to children and families in the state of Indiana. The training 
project involves training Indiana state and county agency 
employees in creative problem solving. 

The design of the evaluation of the training was planned by 
the project coordinator and an independent researcher at ISU. 

The evaluation of the training targets participants' perceptions 
as to whether their behavior has changed as a result of the 
training. It was participants' perceptions of behavior change 
that is measured in this evaluation. It should be noted that the 
measurement of attitudes and perceptions is not equivalent to 
measuring actual behavior. 

The training in creative problem solving is expected to 
provide state and county facilitators with knowledge and skills 
in group problem solving. It is expected that the knowledge and 
skill acquired will be applied to problem solving efforts in the 
work setting (e.g., in inter-agency staffings) . Ultimately, it 
is hoped that the training will positively impact children and 
families who are receiving services from the coordinating 
agencies in the state of Indiana. 

The goal of the training project is to enable Indiana State 
and county agency employees to 

• "apply effective problem solving and solution 
finding principles to direct and immediate 
educational challenges." 

The questions posed are; Will participants rate their use of 
group problem solving and collaboration skills higher on the 
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posttest following the training than prior to the training (i.e., 
than on the pretest)? and Will participants continue to rate 
their group problem solving skills high on the second posttest? 



Methodology 

Participants 

While ninety-two state and county agency employees will 
participate in the training over the course of four training 
periods, only twenty-one participants are included in the 
preliminary analyses reported in this paper. These twenty-one 
participants are child service coordinators for the state of 
Indiana and counties within Indiana. They were invited to 
participate in the training on a voluntary basis given their 
employment in state agencies providing child and family services. 
These individuals were targeted for participation given their 
positions and ability to achieve the goals of the Indiana 
Collaboration Project. Since the participants serve as 
coordinators and facilitators in Step Ahead Councils, a high 
level of competence in leading and facilitating groups was 
assumed. 

Training 

The training in creative problem solving provided "thinking 
tools to use in understanding problems and opportunities, 
generating ideas, and developing and evaluating potential 
solutions." The training occurred over a five day period. It 
was held in a conference room in a large hotel. The scheduling 
of the daily sessions and a complete agenda are included in 
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Appendix A. Participants were preseated in small groups of 6-7 
individuals, including two table facilitators who had received 
prior training in creative problem solving. 

Each participant received a large bound manual, Creative 
Approaches to Problem Solving, which included research articles, 
eleven chapters on the creative problem solving process, and pull 
out folders to use in the training sessions. Additional handouts 
were provided each day of the training. Ample supplies (post-it 
notes, tape flags, markers, pens and pencils, notepads, masking 
tape, etc.) were also available each day of the training. 

The media used in the presentation of the training included 
an overhead projector, color overheads, multiple flip charts, 
laminated 4' by 5^ color posters, video recorders, and a portable 
stereo cassette player. The instructor and the table 
facilitators also used the following as learning tools during 
group activities: rope, string, pictures, recorded sounds, 

music, handballs, novel toys, and other items. 

The method of instruction included presentations, small 
group activities (i.e., problem solving activities), large group 
activities (i.e., reflection on performance and feedback), 
experiential activities, modeling, role-playing, discussion, and 
skill practice. The training sessions were divided into smaller 
sessions with daily reviews of learning objectives. 

Much planning was involved in the preparation of the 
training sessions prior to the training. The instructors and 
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table facilitators also met each evening following the training 
in debriefing sessions. 

Instrument 

The instrument used in assessing participants' perceptions 
of their group problem solving and collaboration skills was the 
••Facilitator Self-Evaluation. •• The "Facilitator Self-Evaluation" 
contained 34 statements regarding participants' behavior in 
facilitating problem solving groups. The participant was asked 
to respond to the statement on a 4-Point Likert scale. For 
example, statement 1. read "As a group facilitator, I encourage 
individuals to be caring, respectful, empathic, and open with 
each other." The directions read, "Please complete this 
evaluation by indicating the degree (circle the appropriate 
number) to which you most often engage in the following 
activities." The possible responses were" "Always," "Usually," 
"Sometimes," and "Never." Refer to Appendix B for the two page 
"Facilitator Self-Evaluation." 

The instrument was developed specifically for the evaluation 
of the creative problem solving training. Items included on the 
instrument were based on previous research on collaboration in 
groups (Knoff & Riser, 1991; West & Cannon, 1988). While the 
statements appearing on the instrument were based in the research 
literature, the statements appearing on the Facilitator Self- 
Evaluation were reworded and the scales were different (i.e., the 
scale used on the Facilitator Self-Evaluation was different from 
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the scales on the instrximents in the literature) . Nonetheless, 
the constructs underlying the concepts were similar. 

Reliability and Validity of the Instrument . Given the use 
of the self-evaluation questionnaire as a pretest and posttest 
measure, it is essential that the instrument be reliable 
(Phillips, 1991). Without an established reliability indice, the 
results obtained cannot conclusively be considered to be effects 
of the training. Rather, a change in scores may merely be 
attributed to the unreliability of the instrument. 

Two appropriate methods for determining the reliability of 
the Facilitator Self-Evaluation are the split half method and 
inter-item correlations. By splitting the instrument into even 
and odd questions and correlating the two sections, an indice of 
reliability may be obtained. Additionally, correlations can be 
calculated I etween each of the items on the self-evaluation 
questionnaire (i.e., each item is correlated with all other 
items) . 

Research Design 

A control group was not incorporated into the design of the 
project for the first year given that all individuals invited to 
participate would be allowed to do so. 

While the preliminary analyses of the evaluation include 
group 1 only, a later evaluation will include all four groups. 
Refer to Appendix C for the timeline of the pretests, training, 
and posttests for the complete project. The dotted box signifies 
the data to be evaluated in this paper. While group 1 will 
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receive a cotal of five posttests, only the results of the first 
two posttests will be analyzed. 

The participants completed the pretest measure prior to the 
training. Posttest #1 was mailed to participants two months 
following the training, and posttest #2 was mailed to 
participants four months following the training. 

Results 

R eliability of the Facilitator Self-Evaluation 

The small sample on which these preliminary analyses were 
performed was considered too small to calculate a reliability 
indice for the instrument. The reliability indice (s) should be 
calculated prior to the evaluation of the four training groups. 
Pretest Ratings 

Cross tabulations of the pretest data provided a baseline of 
participants' perceptions of their group problem solving and 
collaboration skills prior to the training. As indicated in 
Table 1 in Appendix D, the majority of participants rated 
themselves high (e.g., as 'always' or 'usually' engaging in the 
behavior) in all areas assessed on the Facilitator Self- 
Evaluation before the training began. For example, 90% of all 
respondents responded that they always or usually communicated 
clearly and effectively in groups at the pretest. Similarly, 90% 
of all respondents indicated that they always or usually verified 
and reinforced group ownership of the situation at the pretest. 
The fact that respondents perceived that they already used the 
skills that were assessed on the self-evaluation questionnaire 
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suggests that a ceiling effect may have been operating with this 
particular sample of participants. The use of the 4-Point Likert 
scale, as opposed to a 5-, 6-, or 7-Point Likert scale created 
such an effect with this sample. The result of the ceiling 
effect would be to increase the probability of committing a Type 
II statistical error. 

Analysis of Variance 

Regarding differences in scores from pretest to posttest #1 
and posttest #2, analysis of variance revealed six variables of 
interest. A significant difference was found between tests — 
pretest, posttest #1 and posttest #2 — on participants' 
perceptions of their ability to manage the timing of problem 
solving activities to facilitate mutual decision making (F (2, 

37) = 7.59, p < .01). Refer to the line graph in Figure 2 for 
the group means of each test on item 18. 
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18* As a group 
facilitator I I manage the 
timing of problem solving 
activities to facilitate 
mutual decision making at 
each process stage* 



Mean Ratings of Variable #18 



Pretest 



Posttest #1 
Test Number 



Posttest #2 



Figure 2 . 



Figure 2 illustrates that participants' ratings of their 
perceptions of their ability to manage the timing of problem 
solving activities increased from the pretest to the posttests* 
Means and standard deviations for all items on the instrument 
are presented in table 2 in Appendix E. 

Note that a lower score on the Likert scale represents the 
presence of the behavior (i*e* , the respondent rating herself 
with a **1“ perceives that she "always" engages in the behavior)* 
The legend located to the right of the graph indicates the 
meaning associated with the numbers on the Likert scale (6*g.# a 
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behavior) . 

Pairwise comparisons of the pretest with the posttests were 
not calculated for any Of hte 1-way ANOVAs, despite the 
important of doing so. Pairwise comparisons will be conducted 
in the analysis of the data on all four groups of participants. 
For this preliminary analysis, the direction of the change is 
all that is needed. The significance of the pairwise 
comparisions will be available in the complete analyses. 

A significant difference was also found between the tests 
on participants' perceptions of their ability to recognize that 
successful and lasting solutions require common goals (F (2, 37) 

= 4.99, p < .05) . An increase in ratings was evident from 
pretest to posttest #1, as illustrated in the mean ratings for 
the pretest and posttest #1 in Figure 3. 
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20. As a group facilitator, i 
recognize that successful and 
lasting solutions require 
common goals throughout 
problem solving processes . 



4 = Never 
3 = Sometimes 
2 = Usually 
1 = Always 



Mean Ratings of Variable #20 




Pretest Posttest #1 Posttest #2 
Test Number 



Figure i. 



This increase was not maintained from posttest #1 to posttest 
#2, however. 

Similarly, participants rated their ability to effectively 
use divergent and convergent cjuestions, significantly 
differently from pretest to posttest (F (2, 37) = 4.82, p < 

.05). Figure 4 illustrates the mean ratings per group. 
Examination of the means for each test reveals that participants 
rated themselves higher in terms of problem solving skills at 
posttest #1. The effect held for posttest #2. 
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22. As a group facilitator, 
I effectively use divergent 
and convergent questions. 



4 

3 

2 

1 



Never 

Sometimes 

Usually 

Always 



I Mean Ratings of Variable #22 | 










Figure 4 . 



Significant differences were also found between the three tests 
bn participants' ratings of their ability to foster a "pilot" 
problem solving attitude with contingency plans (F (2, 37) = 
5.89, p < .01). Again, participants rated themselves as more 
skilled at posttest #1 than at the pretest, as evident in Figure 
5. The effect appeared to be maintained at posttest #2. 
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23. As a group facilitator, I foster 
the adoption of a "pilot problem 
solving" attitude with contingency 
plans, that recognize that adjustments 
may need to be made. 




Figure 5 . 



Significant differences were also found between the three tests 
on participants' ratings of their ability to assist in the 
redesign, maintenance, or discontinuance of interventions (F (2, 
37) = 6.20, p <.01). On this item, participants rated 
themselves as more skilled at posttest #1 and at posttest #2 
than at the pretest, as evident in Figure 6. 
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25. As a group facilitator, I 
assist in the redesign, 
maintenance, or discontinuance 
of interventions using 
data-base evaluations. 




Figure 6 .^ 



Lastly, a significant difference was found between tests on 
participants' ratings of their ability to encourage evaluation 
of input, process, and outcome variables (F (2, 36) = 3.84, p < 
.05). Participants rated themselves as more skilled at posttest 
#1 and at posttest #2 than at the pretest, as evident in Figure 
7 . 
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27. As a group facilitator, 
I encourage group evaluation 
of impact of input, process, 
and outcome variables on 
desired outcomes. 



4 = Never 
3 = Sometimes 
2 = Usually 
1 = Always 



Figure 7 . 



VI. Discussion, Conclusions, and Recommendations 
The following factors preclude a discussion of the meaning 
of the results obtained. Obtaining statistical significance 
requires an interpretation of the results as opposed to blind 
acceptance of the "significance* of a phenomenon. Before 
interpreting the meaning of the results obtained, a discussion 
of possible confounds is warranted. Similarly, the reliability 
of the instrument must be established before its use as a 
pre/posttest measure is justified. A discussion of these 
concerns ensues. 
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Mean Ratings of variable #27 




Pretest Posttest #1 Posttest #2 
Test Number 
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Pretest Ratings 

The high self ratings on the pretest present a concern 
regarding the effectiveness of the "Facilitator Self-Evaluation." 
Since participants are expected to increase their skills as a 
result of the training, they may actually increase these skills 
without a significant difference being evident on the instrximent 
due to the arbitrary ceiling. The ceiling effect seems to have 
resulted from three factors: (a) the experience 

leading/facilitating groups this sample appears to possess, (b) 
the restricted range of possible responses (i.e., the 4 point 
Likert scale options) , and (c) the wording of the possible 
responses (e.g., the choice options "never" and "always" are 
absolute and do not represent a continuum of more and less 
skill) . 

Facilitator Self-Evaluation 

D esign of the Instrument . The "Facilitator Self-Evaluation" 
presents a concern. The desired response to each item (i.e., 
question) is readily apparent in that all items and scales are 
slanted in the same direction (i.e., each item represents the 
presence of a skill) . As such, the respondent need not give 
careful attention to each item in rating themselves on each. 

Further considerations are presented in the guidelines on 
instrument development presented in Table 3 below. 
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Table 3 

Guidelines on Inatrument Development 



1. Respondents should be assured of confidentiality 

2. Respondents should be assured that answers are neither 
correct nor incorrect 

3. The directions on the instrument should be understandable 

4. Instructions should include definitions of choice options 
(e.g., a definition of "most of the time* should be 
included in the instructions) 

5. An estimate of the time to complete the instrument should 
be provided 

6. Wording should be simple, clear, straight forward, and to 
the point 

7. The instrument should be easy to read and attractive in 
terms of format and spacing 

8. Items should be included in categories so that respondents 
do not have to change their point of reference 

9. The favorable end of the scale should be varied so that 
the respondent must give careful attention to each item 
rather than merely responding without weighing the 
statement 

10. Only one idea should be presented per item 

11. Absolute terminology (e.g., "all" or "none") should be 
avoided 



raT iabilitv . AS indicated, the reliability of the 
instrument was not calculated due to the small sample size. It 
should however, be calculated before the data from all four 
groups is analyzed. 

Validity . There are several possible threats to the 
internal validity of the study. Certain uncontrolled factors 
may have directly affected the participants' ratings on the 
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self-evaluation posttests and hence obscured the actual training 
effects. Internal validity refers to whether the study is 
measuring what it purports to measure. One possible confound in 
such a pretest-posttest design is maturation effects (Patrick, 
1992) . Maturation pertains to changes in performance due to 
natural growth occurring over time. Since the participants were 
adults who had advanced graduate training, it is unlikely that 
maturation had a significant affect on performance over the six 
months of the study conducted thus far. 

A second confound may occur when participants are posttested 
more than once in the pretest-posttest design. In this case, the 
pretest may have a transfer effect, positive or negative, on the 
subsequent posttest. In other words, participants may become 
practiced at being assessed. Such a confounding effect cannot be 
ruled out. 

Third, another possible confound is the reactive effect of 
the training and the participants' knowledge that they were 
participating in an evaluation. This effect is otherwise known 
as the Hawthorne Effect. Hence, it may be the novelty of the 
training that produces the effect rather than the training 
itself. 

These possible confounds can be eliminated in the future by 
the use of a control group. In order to reduce these effects 
through the use of a control group, random assignment of 
participants to the training and control groups is necessary. 
Figure 8 illustrates this experimental design. 
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Pretest Training Posttest 


Training /\ 






Group V y / \ 


Control 




Group \^J 



Figure 8. Experimental Design Employing a Standard. No- 
Treatment Control 

Alternatively, an experimental design, which would be 
easier to implement in an applied setting, might include a 
second group that received the training and posttests only. See 
the depiction of this design in Figure 9. 




Figure 9. Experimental Design Employing Two Treatment Groups 



An analysis of the differences between the posttests for 
the two groups would reveal any testing confounds caused by 
multiple administrations of the Facilitator Self-Evaluation. 
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These designs would, of course, be more complex given the number 
of training groups involved in the training project. 

Aside from the highly specific group from which participants 
were selected is the concern over the possible differences 
between those selected who were accessible, (i.e., able to attend 
the training) and those selected who were not accessible (i.e., 
who did not attend the training) . The individuals who 
participated in the training may have differed on particular 
characteristics from those who were selected but did not 
participate. 

Conclusion 

As discussed earlier, the purpose in evaluating a training 
program may be to investigate the effects of training variables 
in a research approach. For example, using an experimental 
design in evaluating future creative problem solving trainings 
would allow for the investigation of the following variables; 
the effects on the duration of the training (e.g., 3-day training 
versus 5-day training) , the effects of follow-up training, and 
the effects of participant characteristics. 

The obvious limitation to this evaluation is that the 
instrument used measures perceptions as opposed to actual 
behaviors. And, perceptions do not necessarily correlated with 
behaviors. To measure participants' behaviors as a result of the 
training would entail a more complex evaluation design involving 
others' ratings (e.g., supervisors) of behavior and/or 
observations of behavior. Given the goal of the training 
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project, the measurement of participants' performance on the job 
namely their collaboration and problem solving skills, would be a 
desirable addition to the evaluation design. 
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Appendix A 



Day 1 

Morning Session SsOQ a.m. — z — 1.2 tfl Q NQQH 

• Welcome, Introductions, and Overview 

• Exploring Your Personal Creative Process 

• Introduction to Creative Problem Solving 

• Conceptions and Misconceptions of Creativity 

• Personal Orientation: Examining Your Creativity Style 

Afternoo n Session ^ 1 ;QQ P il B . ~ 6;QQ 

• Guidelines for Creative Problem Solving 

• Preparing to Learn and Apply Creative Problem Solving 

• Introduction to the Understanding the Problem Component 

• Examining Stages and Tools for Identifying and Understanding 
Problems or Opportunities 



Day 2 



Morning ^ 8 : QQ a tlfl, z 12 i Qi2 — IlPPU 

• Introduction to the Generating Ideas Component 

• Examining and Applying Tools for Divergent Thinking 

• Criteria and Creativity 

Afternoon Session 1;QQ Pitfli — z — 5;QQ PiKl t. 

• Introduction to the Planning for Action Component 

• Examining Tools for Developing Solutions and Planning Change 



Day 3 

Morning Session ^ 8:00 z 12;QQ NPQD 

• Qualities of an Effective Facilitator 

• Conceptions of Leadership 

• Debrief the Leadership Inventory Profile 

Afternoon Session 1 ; QQ J2iJIL — z — 5 ; Q Q P i III 

• Creative Problem Solving Tool Overview 

• Task Appraisal and Process Planning 
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Day 4 










Mornina Session 






8:00 a.m 


- 


12:00 


Noon 


• Practice Creative 
Debriefing 


Problem 


Solving 


Facilitations 


and 






Afternnnn Session 






1:00 XL. 


m. - 


6:00 




• Practice Creative 
Debriefing 


Problem 


Solving 


Facilitations 


and 












Day 5 










Mornino Session 






8:00 a.m. - 


12:00 


Noon 


• Practice Creative 
Debriefing 


Problem 


Solving 


Facilitations 


and 






AftArnnnn Session 






1:00 P. 


,m. - 


• 4:00 


P.m. 


• Ethics of Creative Problem Solving Facilitation 

• Taking it Forward 

• Closing Comments and Program Evaluation 
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Appendix B 



Facilitator Self*Evaluation 
S. Frantz & L. Bischoff 

Please c<Nnplete this evaluation by indicating the degree (circle the appropriate number) to which you most often 



engage in the following activities. 

As a group facilitator, I... 

1 . encourage ittdividu^ to be caring, respectful, empathic, 
and <^>en with each other. 

2. establish, maintain, and facilitate report among all 
individuals involved. 

3 . maintain a positive self-concept and enthusiastic 
attitude. 

4. am willing to learn firom the group and encourage the 
group to learn from each other. 

5 . am able to manage my stress and group stress. 

Sa. am flexible when the une?q)ectedh^pens. 

6. remain ^)proachable throughout the group process. 

7. re^)ect and encourage ie^)ect of divergent viewpoints. 

8 . communicate dearly and effectively. 

9 . utilize effective communication skills, including active 
listening, questioning, and paraphrasing. 

10. invite and encourage all individuals involved in the 
problem solving relationshq) to participate. 

10a assist groups in framing and wmking through stages 
of group devek^ent. 

1 1 . adjust my skills to accommodate the experience of 
group members. 

12. effectively interpret the nonverbal communication of 
myself arid oth^. 

1 3 . effectively elicit information, share information, 
explore problems, set goals, and objectives. 

1 3a summarize and clarify information flow. 

1 3b. question to promote content and process clarity. 

14. pursue issues with ai^rropriate persistence, even if a 
level of discomfort is sqrparent. 

14a reinforce a productive working environment. 

1 5. give, solicit, and reinforce continuous feedback that is 
specific, immediate, and objective. 

15b. make tqrprtqwiale changes in my behavior based on 
constructive feedback. 

16. give credit and encourage consensual credit to others 
for their ideas and accomplishments. 

1 7 . manage conflict and confiontation skillfully to 
maintain group relationships. 

18. manage the timing of problem solving activities to 
facilitate mutual decision making at each process stage. 

18a. establish time-lines for problem solving sessions. 

1 8b. provide clear task priorities and session objectives. 

19. am willing to say and am open to hear “I don’t 
know...let’s find out.” 



Always Usually Sometimes Never 



2 

2 

2 

2 

2 

2 



2 

2 

2 

2 

2 



2 

2 

2 



3 

3 

3 

3 

3 

3 



3 

3 

3 

3 

3 



3 

3 

3 



4 

4 

4 

4 

4 

4 



4 

4 

4 

4 

4 



4 

4 

4 
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1 


Facilitator Self-Evaluation 








1 


(Page 2) 










1 


Always Usually Sometimes Never 


1 As a 


group facilitator, I... 

recognize that successful and lasting solutions require 










1 20. 










1 


common goals throughout problem solving processes. 


1 


2 


3 


4 


1 20a. 


check with the group to verify and reinforce 
group ownersh^ of situation and related content. 


1 


2 


3 


4 


1 20b. 


check with the group during the session 
for apprc^triateness'of direction. 


1 


2 


3 


4 


1 21. 


generate and encourage others to generate viable 
alternatives through toiinstorming techniques 










1 


characterized by active listening, nonjudgmental 
responding, and tppropriate consequences. 


1 


2 


3 


4 


1 22. 


effectively use divergent and convergent questions. 


1 


2 


3 


4 


1 23. 


foster the ad(^)tion of a “pilot problem solving” 
attitude with contingency plans, that recognize 
that adjustments may need to be made. 


1 


2 


3 


4 


1 


remain available throughout the group process for 
support, modeling, and/or assistance in 
intervention modification. 


1 


2 


3 


4 


1 2S. 


assist in die redesign, maintenance, or discontinuance 
of interventions using data-base evaluation. 


1 


2 


3 


4 1 


1 26. 


utilize continuous fe^back to help the group 
maintain, revise, or terminate group activities. 


1 


2 


3 


4 1 


1 27. 


encourage group evaluation of impact of input, 
process, and outcome vari^ies on desired outcomes. 


1 


2 


3 


4 


1 28. 


facilitate equal oppcxtunities by showing respect for 
individual differences in appearance, race, sex, disability, 
ethnicity, religion, socioeconomic status, and/or ability. 


1 


2 


3 


4 1 


1 29. 


advocate for services that will accommodate the social, 
educational, and vocational needs of children and families. 


1 


2 


3 


4 1 


1 30. 


utilize the principles of least restrictive environment 
in decisions regarding services for families and children. 


1 


2 


3 


4 1 


1 31. 


ensure that persons involved in planning and 
implementing the problem sdving processes 
are also involved in its evaluation. 


1 


2 


3 


4 


1 32. 


self-evaluate my strengths and weaknesses to modify 
my behavicx influencing problem solving processes. 


1 


2 


3 


4 1 


1 33. 


remain interested in the needs and concerns of the group 
and systems within which the group is working. 


1 


2 


3 


4 


1 34. 


am trustwrathy, honest, and maintain confidentiality. 


1 


2 


3 


4 


1 


tcAkfcnn S G.. Dnrval. K. B.. A Treffincer* D. J. fl994^. Creative Approaches 


to Problem Solving. 


1 Dubuque, Iowa: Kcndall/Hunt Publishing Company. 












Knoff, H., McKenna, A., & Riser, K. (1991). Toward a consultant effecUveness scale: InvesUgatmg 


1 the characteristics of effective consultants. School Psvcholojrv Review. 20. 81-96. 








West, J. F. & C^non, G. S. (1988). Essential collaborative consultation competencies for regular and 


1 special educators. Joumal of Leaminc Disabilities. 21 (1). 56-63. 
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1S54- 



•- 



1555 



Lsgend: 

Training 

□ 



Posttest 



Sep Oct Nov Bee Jan Feb Mar Apr May Jun Jul Aug Sep 






4 


— GROUP 1 -| 5 1 


2 


— GROUP 2 -| 3 \ 


1 


— GROUP 3 — j 1 1 






GROUP 4 



O 

ERIC 



3 ii 
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Appendix D 

Table 1 

Pretest Frequencies 



Item # 


"Always" 
n = 


% 


"Usually 
n = 


■ 

% 


"Always/Sometimes" 

k_= i 


1 . 


6 


29 


13 


62 


19 


90 


2 . 


6 - 


29 


11 


52 


17 


81 


3 . 


8 


38 


12 


57 


20 


95 


4 . 


12 


57 


8 


38 


20 


95 


5a * 


3 


14 


12 


57 


15 


71 


6 . 


7 


33 


11 


52 


18 


86 


7 . 


8 


3 8 


13 


62 


21 


100 


8. 


6 


29 


13 


62 


19 


90 


9 . 


3 


33 


12 


57 


15 


71 


10 . 


6 


29 


11 


52 


17 


81 


10a. 


8 


38 


11 


52 


19 


90 


11. 


3 


14 


11 


52 


14 


67 


12. 


2 


10 


13 


62 


15 


71 


13 . 


1 


5 


14 


67 


15 


71 


13a. 


2 


10 


13 


62 


15 


71 


13b. 


4 


19 


8 


38 


12 


57 


14 . 


2 


10 


11 


52 


13 


62 


14a. 


2 


10 


12 


57 


14 


67 


15. 


5 


24 


12 


57 


17 


81 


15b. 


1 


5 


10 


48 


11 


52 


16. 


3 


14 


16 


76 


19 


90 


17 . 


9 


43 


9 


43 


18 


86 


18. 


2 


10 


12 


57 


14 


67 


18a. 


0 


- 


10 


48 


10 


48 


18b. 


5 


24 


9 


43 


14 


67 


19 . 


6 


29 


9 


43 


15 


71 


20 . 


14 


67 


6 


29 


20 


95 


2 0a . 


3 


14 


16 


76 


19 


90 


20b. 


3 


14 


10 


48 


13 


62 


21 . 


3 


14 


13 


62 


16 


76 


22 . 


3 


14 


11 


52 


14 


67 


23 . 


0 


- 


7 


33 


7 


33 


24 . 


2 


10 


6 


29 


8 


38 


25. 


5 


24 


11 


52 


16 


76 


26. 


0 


- 


5 


24 


5 


24 


27. 


4 


19 


8 


38 


12 


57 


28. 


2 


10 


8 


38 


10 


48 


29. 


12 


57 


8 


38 


20 


95 


30 . 


14 


67 


6 


29 


20 


95 


31. 


, 17 


81 


3 


14 


20 


95 


32 . 


7 


33 


7 


33 


14 


67 


33 . 


5 


24 


10 


48 


15 


71 


34 . 


6 


29 


14 


67 


20 


95 



o 3 ‘1 

ERIC 



t 
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Table 3 

Pretest and Post test Means and Standard D eviations 





Pretest 


Posttest #1 


Posttest #2 


Item 


X 


SD 


X 


SD 


X 





1. 


1.8 


6.1560 


1.4286 


0.5136 


1.4286 


0.5345 


2. 


2.0 


0^.6489 


1.5714 


0.5136 


2.0 


0 . 5774 


3. 


1.65 


0.5871 


1.4286 


0.5136 


1.7143 


0.4880 


4. 


1.4 


0.5026 


1.4286 


0.5136 


1.4286 


0.5345 


5. 


2.15 


0.6708 


2.0 


0.5547 


1.7143 


0.7559 


5a. 


1.8 


0.6959 


1.5714 


0.5136 


2.0 


0.0000 


6. 


1.6 


0.5026 


1.4286 


0.5136 


1.7143 


0.7559 


7 . 


1.6842 


0.4776 


1.3846 


0.5064 


1.7143 


1.1127 


8. 


2.15 


0.6708 


1.7857 


0.5789 


1.8571 


0.3780 


9. 


1.9 


0.7182 


1.7857 


0.4258 


2.0 


0 . 5774 


10. 


1.7 


0.6569 


1.4286 


0.5136 


1.4286 


0.5345 


10a. 


2.25 


0.7864 


1.7857 


0.6993 


1.7143 


0.7559 


11 . 


2.1579 


0.6021 


1.8571 


0.6630 


1.5714 


0.5345 


12 . 


2.25 


0.5501 


2.0 


0.5547 


2.0 


0.5774 


13 . 


2.2105 


0.7133 


1.9286 


0.4746 


1.5714 


0.5345 


13a. 


2.1579 


0.7647 


1.7857 


0.5789 


1.8571 


0.6901 


13b. 


2.2105 


0.6306 


1.7857 


0.8018 


1.5714 


0.5345 


14 . 


2.3 


0.7327 


2.2857 


0.7263 


2.2857 


0.7559 


14a . 


1.9474 


0.7799 


1.5 


0.5189 


1.5714 


0.5345 


15. 


2.5 


0.6882 


2.0 


0.7845 


1.8571 


0.6901 


15b. 


1.95 


0.5104 


1.7857 


0.8018 


1.8571 


0.6901 


16. 


1.65 


0.6708 


1.50 


1.5189 


1.4286 


0.5345 


17 . 


2.25 


0.6387 


1.9286 


0.6157 


2.0 


0 . 5774 


18. 


2.5789 


0.6070 


2.0714 


0.4746 


1.7143 


0.4880 


18a . 


2.1 


0.7881 


1.8571 


0.5345 


2.0 


0.5774 


18b. 


2.0 


1.7947 


1.4615 


0.5189 


1.5714 


0.5345 


19. 


1.3 


0.4702 


1.3571 


0.4972 


1.7143 


0.7559 


20. 


1.85 


0.4894 


1.3571 


0.4972 


2.0 


0.6325 


20a. 


2.2 


0.6959 


1.7143 


0.4688 


1.8333 


0.7528 


2 0b. 


2.05 


0.6048 


1.7857 


0.5789 


1.6667 


0.5164 


21. 


2.2 


0.7678 


1.6429 


0.6333 


1.8333 


0.4082 


22. 


2.7 


0.5712 


2.0714 


0.6157 


2.1667 


0.7528 


23 . 


2.6316 


0.8951 


1.7857 


0.4258 


2.0 


0.6325 


24 . 


1.95 


0.6863 


1.6154 


0.6504 


1.6667 


0.5164 


25. 


2.95 


0.6863 


2.2857 


0.7263 


2.0 


0.6325 


26. 


2.2 


0.7678 


1.7857 


0.5689 


1.8333 


0.4082 


27. 


2.3684 


0.6840 


1.8571 


0.6630 


1.6667 


0.5164 


28. 


1.45 


0.6048 


1.4286 


0.5136 


1.6667 


0.5164 


29 . 


1.4 


0.5982 


1.4286 


0.5136 


1.1667 


0.4082 


30. 


1.35 


0.7452 


1.3571 


0.4972 


1.1667 


0.4082 


31. 


2.0 


0.9177 


1.50 


0.5189 


1.6667 


0.5164 


32. 


2.0 


0.7454 


1.4286 


0.6462 


2.0 


0.6325 


33 . 


1.7 


0.4702 


1.4286 


0.5136 


1.3333 


0.5164 


34 . 


1.4 


0.5982 


1.1429 


0.3631 


1.1667 


0.4082 




3o 



